Multithreading and Thread Migration Using Mpi and Myrinet

The balance between CPU speed and interconnection network throughput in distributed memory parallel computers varies with each generation of systems, but the trend is that CPUs are gaining performance faster than the interconnection networks. This means that remote data accesses are becoming more expensive relative to local accesses in terms of CPU cycles. Therefore, remote memory access mechanisms that were suited to a previous generation of parallel machines may be less appropriate for current clusters. This research evaluates a multithreaded programming paradigm with cached remote memory accesses and thread migration to exploit array locality on a cluster with Myrinet. The approach, called Nomadic Threads, was originally developed for the CM5, but has been adapted to use MPI on Linux clusters. The results show that the current surfeit of CPU power vs. network throughput dramatically changes scaling characteristics of some programs while others behave much as they did on the decade-old CM5.